Generating, collecting, analyzing and consuming test data are keys to business intelligence for companies. Managing the creation of test plans, test automation, test data and then utilization of very expensive assets becomes a discipline and process of its own. The tests are often too complex to do manually and far too time consuming to be executed in a manufacturing environment. This testing process is not only time consuming but also resource intensive and thereby negatively impacting impacts the overall system design productively. The framework of database and test tools wrapped around this process is referred to as a test automation framework. Test automation is a key component of test processes and critical to enhancing the productivity of the overall team. Such infrastructure provides these companies with a competitive advantage in systematically defining test plans and then quickly achieving automation and generating large amounts of intelligent product critical data.
Automatic Testing Equipment (ATE) industry has pushed to develop a framework which would support the sharing of test information, data and results across various enterprise platforms. Usually the test engineer develops a test plan comprising of design of the testbed, equipment, test specification and other testing requirements. Based on the test plan, the test equipment is assembled and testing team orchestrates the automation requirements and further collects, analyzes and reports the data upon test completion. Such a process of developing tool sets that automate the test activity is known as business process automation.
The test process for any given company can encompass test plan development, equipment selection, automation software development, data collection and storage, data analysis, test report/data sheet assembly and publication, supply chain test monitoring (contract manufacturing), test station utilization, test station reservation and scheduling, test asset calibration and tracking, and manufacturing execution system (MES).
Technologically, the testing system and their framework can be classified into three main groups of systems. The first one represents the multiple standalone station which do not share any data, computing resources (such as server, data storage among others), test & data formats. These systems are highly fragmented and have a significantly different test data and process format from one test station to another. This configuration represents fragmentation to the extreme end where the test data format may vary from one test-station to another test-station within the same group resulting in an extremely inefficient system.
The second group of testing systems and framework include a system with its own dedicated file server where the test data and the test plans are centrally deposited and stored within an organization or a group. This obviously provides a more centralized approach to the standalone system as described above. In such configuration test plans typically begin with engineers developing text to spreadsheet plans with various individual formats and storage locations. These then translate into individual automation plans and automation solutions with minimal amounts of reusability. In both above cases, once automation is developed and in place, the production, storage, analysis and publications of results ranges from local text files to haphazardly organized file shares.
The third approach is a client-server based system for configuring and managing the test station and test data and process. This involves a structured and organized approach to managing the data and its format for providing higher degrees of interoperability to users within the same group or organization. The structured approach gains efficiencies in structuring data however creates a very proprietary and investment intensive system. The system in such scenario may not be interoperable within geographically dispersed group/organization and will definitely not be interoperable across various organizations.
Each of these approaches introduces several challenges and therefore inefficiencies within the organization. Each of these traditional methods means internal investment of proprietary systems which, one at a time, provide business process automation to the phases above. Companies have previously followed an evolutionary path to address this process. Such unstructured management of test plans and handling of test data and results leads to productivity losses due to issues related to data portability, data organization and absence of a common platform for sharing the data within organizations and among other organizations. This creates confusion in organization of results as well as communication and publication challenges. Format of result publications and analysis reports vary from one organization to another and in many cases from one group within an organization to another group within the same organization. The absence of a unified standards and process for such test data sharing and management leads to a challenge in data management.
The inefficiencies and absence of a unified process have led to a need for an automated enterprise test system framework which allows various organizations to share data across multiple test platforms. To further leverage this unified environment a configurable test-case library can be developed and shared across several platform's based on test equipment specifications and Design Under Test (DUT) requirements. Users may then use existing test-cases and avoid developing the specific test-cases for their design thereby, adding efficiency to this overall process. Automatic Test Markup Language (ATML) has been recognized as a standard which provides XML based test information exchange among various ATEs making them interoperable.